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1. 


Introduction 


The  term  digital  watermarking  refers  to  the  process  of  embedding  a  “secret”  digital  signal  (hidden 
message)  in  another  digital  signal  (image  or  audio)  called  “cover”  or  “host.”  Two  applications  of 
watermarking  are  a)  image/audio  authentication  and  b)  steganography,  which  is  an  attempt  to  establish 
covert  communication  between  trusting  parties. 

The  first  step  in  the  design  of  a  watermarking  system  is  the  embedding  process.  This  is  a  crucial 
task  since  watermarking  properties  and  detector  design  and  performance  depend  directly  on  the  way  the 
watermark  is  inserted  within  the  host  data.  While  each  specific  watermarking  application  has  its  own 
requirements  [1],  [2],  the  broad  objective  of  most  steganographic  applications  is  a  satisfactory  trade-off 
between  hidden  message  resistance  to  noise/disturbance,  information  delivery  rate,  and  host  distortion. 
Message  embedding  can  be  performed  either  directly  in  the  time  (audio)  or  spatial  (image)  domain 
[3]- [6]  or  in  a  transform  domain  (for  example,  for  images  we  may  consider  full-frame  discrete  Fourier 
transform  (DFT)  [7]- [10] ,  full-frame  discrete  cosine  transform  (DCT)  [11],  block  DFT  or  DCT  [12],  [13], 
or  wavelet  transforms  [14]- [16] ) .  Direct  embedding  in  the  original  host  signal  domain  may  be  desirable 
for  system  complexity  purposes,  while  embedding  in  a  transform  domain  may  take  advantage  of  the 
particular  transform  domain  properties  [17]. 

In  this  present  work,  we  focus  our  attention  on  transform  domain  spread-spectrum  (SS)  embedding 
methods  for  image  steganography.  Once  the  transform  embedding  domain  has  been  selected,  the  hidden 
message  can  be  applied  to  the  host  data  through  an  additive  or  multiplicative  rule  [5]- [8],  [12],  [18].  In  the 
literature,  additive  spread-spectrum  embedding  methods  use  an  amplitude  modulated  pseudorandom 
signature  to  deposit  one  information  symbol  across  a  group  of  host  data  coefficients  [5],  [7],  [12]  or 
a  linearly  transformed  version  of  the  host  data  coefficients  [18].  In  multiplicative  rule  SS  embedding, 
message  data  multiply  host  data  coefficients  [8]. 

Spread-spectrum  embedding  algorithms  for  blind  image  steganography  (that  is,  hidden  message 
recovery  without  knowledge  of  the  original  image)  have  been  based  on  the  understanding  that  the 
host  signal  acts  as  a  source  of  interference  to  the  secret  message  of  interest.  Yet,  it  should  also  be 
understood  that  this  interference  is  known  to  the  message  embedder.  Such  knowledge  can  be  exploited 
appropriately  to  facilitate  the  task  of  the  blind  receiver  at  the  other  end  and  minimize  the  recovery  error 
rate  for  a  given  host  distortion  level,  minimize  host  distortion  for  a  given  target  recovery  error  rate, 
maximize  the  Shannon  capacity  of  the  covert  steganographic  channel,  etc.  Indeed,  if  we  were  to  place  the 
steganography  application  in  an  information  theoretic  context,  it  could  be  viewed  as  a  communications- 
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with-side-information  problem  [19]- [21] .  Optimized  embedding  methods  can  facilitate  host  interference 
suppression  at  the  receiver  side  when  knowledge  of  the  host  signal  is  adequately  exploited  in  system 
design. 

In  this  paper,  for  any  given  image,  (block)  transform  domain,  and  host  bins,  we  derive  the  additive 
embedding  signature  that  maximizes  the  output  signal-to-interference-plus-noise  ratio  (SINR)  of  the 
linear  maximum  SINR  receiver  filter.  We  establish  that,  under  a  (colored)  Gaussian  assumption  on  the 
host  bins,  this  same  signature  minimizes  the  receiver  bit  error  rate  (BER)  at  any  mean  square  (MS) 
image  distortion  level,  minimizes  -conversely-  the  MS  image  distortion  at  any  target  BER,  and  maximizes 
the  Shannon  capacity  of  the  covert  link.  We  then  generalize  our  findings  to  cover  joint  signature  and 
linear  host  data  projection  optimization  along  the  same  lines.  In  this  present  work,  we  consider  only 
scalar  parameterized  host  data  projection  as  in  [18].  Finally,  we  extend  signature-only  as  well  as  joint 
signature  and  host-projection  optimization  to  multiuser  (multiple  signature)  embedding.  Our  emphasis 
is  directed  primarily  toward  low  complexity,  sequential,  conditional  multiuser  optimization. 

The  rest  of  the  paper  is  organized  as  follows.  Section  II  presents  our  core  signature  and  embedding 
optimization  results.  These  results  are  generalized  to  multiple  signature  embedding  in  Section  III. 
Section  IV  is  devoted  to  experimental  studies  and  comparisons.  A  few  concluding  remarks  are  given  in 
Section  V. 

2.  Signature  Optimization  for  Spread-Spectrum  Embedding 

In  this  section  we  develop  an  optimized  spread-spectrum  (SS)  steganographic  system.  To  draw  a 
parallelism  with  conventional  SS  communications  systems,  in  SS  watermark  (message)  embedding  the 
watermark  can  be  regarded  as  the  SS  signal  of  interest  transmitted  through  a  noisy  “channel”  (the 
host).  The  disturbance  to  the  SS  signal  of  interest  is  the  host  data  themselves  plus  potential  external 
noise  due  to  physical  transmission  of  the  watermarked  data  and/or  processing/attacking.  The  purpose 
of  the  watermark  detector  is  to  withstand  the  influence  of  the  total  end-to-end  disturbance  and  recover 
the  original  hidden  message. 

A.  Signal  Model  and  Notation 

Consider  a  host  image  H  £  NiNlxN2  that  is  to  be  watermarked  where  A4  is  the  image  alphabet  and 
N\  x  N2  is  the  image  size  in  pixels.  Fig.  1(a)  shows  a  grey  scale  baboon  image  example  in  J\/lNlxN2  = 
{0,1,- ••  , 255} 256x256 .  Without  loss  of  generality,  the  image  H  is  partitioned  into  P  local  blocks  of 
size  AlpAl>  pixels.  Each  block  Hi,H2,--  ,Hp  is  to  carry  one  hidden  information  bit  bp  £  {±1},  p  = 
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Fig.  1.  (a)  Baboon  image  example  H  £  {0, 1,  •  •  •  ,  255} 256x 256 .  (b)  Host  data  autocorrelation  matrix 
(8x8  DCT,  63-bin  host). 
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1,2,  •  •  •  ,  P,  respectively.  Embedding  is  performed  in  a  real  2-dinrensional  transform  domain  T.  After 

N1xN2 

transform  calculation  and  conventional  zig-zag  scanning  vectorization,  we  obtain  T( Hp)  G  M  p  , 
p  =  1,2,- ••  ,P.  From  the  transform  domain  vectors  T( Hp)  we  choose  a  fixed  subset  of  L  <  Nl*N2 
coefficients  (bins)  to  form  the  final  host  vectors  xp  G  ML,  p  =  1,2,  ••  •  ,P  (for  example,  it  is  common 
and  appropriate  to  exclude  the  dc  coefficient  T(Hp)[l]  from  the  host  xp). 

The  autocorrelation  matrix  of  the  host  data  x  is  an  important  statistical  quantity  for  our  develop¬ 
ments  and  is  defined  as  follows: 

R*  =  £{xxr}  =  pJ^XpXp  (1) 

p=  i 

where  E  {•}  denotes  statistical  expectation  (here,  with  respect  to  x  over  the  given  image  H)  and  T  is 
the  transpose  operator.  It  is  easy  to  verify  that  in  general  RT  ^  cuIl,  a  >  0,  where  II  is  the  size-L 
identity  matrix;  that  is,  R^  is  not  constant-value  diagonal  or  “white”  in  field  language.  For  example, 
8x8  DCT  with  63-bin  host  data  formation  (exclude  only  the  dc  coefficient)  for  the  baboon  image  in 
Fig.  1(a)  gives  the  host  autocorrelation  matrix  R^,  in  Fig.  1(b). 

B.  Signature  Optimization 

Consider  direct  additive  SS  embedding  of  the  form 


y  =  Abs  +  x  +  n 


(2) 


where  A  >  0  is  the  bit  amplitude,  s  G  ML,  ||s||  =  1,  is  the  (normalized)  embedding  signature  to  be 
designed,  and  n  ~  A f  represents  potential  external  white  Gaussian  noise1  of  variance  cr^.  The 

mean  squared  distortion  of  the  original  image  due  to  the  watermark  only  is 


V  =  f|||A6s  +  x-x||2}  =  A2. 


(3) 


With  signal  of  interest  Abs  and  total  disturbance  x  +  n  in  (2),  the  linear  filter  that  operates  on  y 
and  offers  maximum  SINR  at  its  output  is 

E  |  ||wt(A6s)||2|  ^  _1 

wmaxsiNR  =  arg  max — f - 2_r  =  (Rx  +  s.  (4) 

w  E  I ||wr(x  +  n)||2 1 

The  exact  maximum  output  SINR  value  attained  is 


F{||sT(R,  +  u2lL)-1A6s||2) 

SINRmax  =  1  - =  ^S(R-  +  °J-l)  s. 

E  |||sT(Ra;  +  allL)  1(x  +  n)||“| 


xAdditive  white  Gaussian  noise  is  frequently  viewed  as  a  suitable  model  for  quantization  errors,  channel  transmission 
disturbances,  and/or  image  processing  attacks. 
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We  propose  to  view  SINRmax  as  a  function  of  the  embedding  signature  s,  SINRmax(s),  and  identify 
the  signature  that  maximizes  the  SINR  at  the  output  of  the  maximum  SINR  filter.  Our  findings  are 
presented  in  the  form  of  a  proposition  below.  The  proof  is  straightforward  and,  therefore,  omitted. 


Proposition  1  Consider  additive  SS  embedding  according  to  (2).  Let  qi,q2,-  ,  q l  be  eigenvectors  of 
Rx  in  (1)  with  corresponding  eigenvalues  X\  >  A2  >  •  •  •  >  A l-  For  any  watermark  induced  distortion 
level  V ,  a  signature  that  maximizes  the  output  SINR  of  the  maximum  SINR  filter  is 


sopt  =  arg  max  • 

S 


|^2sr  (Rx  +  <r2nlL)  1  s|  =  c\l- 
When  s  =  q^,  the  output  SINR  is  maximized  to 

SINRmax(  qL)  =  A\l(Rx  +  a2nlL)-\L  = 
and  maximum  SINR  data  filtering  simplifies  to 


V 


A2 


A  L  +  A  l  +  a2 


wmaxsiNRy  =  q£  (Rx  +  <%W  V  =  qly-  n 


(6) 


(7) 


(8) 


In  summary,  Proposition  1  says  that  the  “minimum”  eigenvector  of  the  host  data  autocorrelation  matrix, 
when  used  as  the  embedding  signature,  sends  the  output  SINR  to  its  maximum  possible  value  ■ 

At  the  same  time,  maximum  SINR  filtering  becomes  plain  signature  (eigenvector)  matched  filtering. 

If,  in  addition,  we  are  allowed  to  assume  that  x  is  Gaussian,  x  ~  J\f  (0,  Rx),  then 


b  =  sign  ( w^axSINRy ) 

is  the  optimum  (minimum  probability  of  error)  bit  detector  [22]  with  probability  of  error 


Pe  =  Q[  AJsT(Rx  +  a2nlLy\  =  Q  [  v/SINRmax(s 


(9) 


(10) 


where  Q(a)  =  /a°°  dr.  We  see  that  Pe  is  a  monotonically  decreasing  function  of  SINRmax. 

If  we  now  view  Pe  as  a  function  of  the  embedding  signature  s,  Pe(s),  then  Proposition  2  below  follows 
directly  from  Proposition  1  and  (10). 

Proposition  2  Consider  additive  SS  embedding  according  to  (2)  with  x  ~  jV(0,  R^).  Let  qi,  q2,  •  •  •  ,  q l 
be  eigenvectors  of  RT  with  corresponding  eigenvalues  Ai  >  A2  >  •  •  •  >  Xl-  For  any  watermark  induced 
distortion  level  V,  a  signature  that  minimizes  the  probability  of  error  of  the  optimum  bit  detector  is 


sopt  =  arg  min  <Q[A\I  sT(R.x  +  a2 1 


s  =qt. 


(ii) 
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When  s  =  q^,  the  probability  of  error  of  the  optimum  detector  is  minimized  to 


Pei  qi) 


(12) 


and  optimum  detection  reduces  to 

b  =  sign  (q^y)  •  (13) 

Conversely,  for  any  preset  probability  of  error  level  Pe,  s  =  q^  minimizes  the  watermark  induced 
distortion  to 

V=(XL  +  al)[Q~1(Pe)]2.  □  (14) 


Proposition  2  explains  that  under  a  Gaussian  host  data  assumption  the  “minimum”  eigenvector  of  the 
host  data  autocorrelation  matrix,  when  used  as  the  embedding  signature,  allows  message  recovery  with 
the  minimum  possible  bit  error  rate  Q  [\J  and  trivial  signature  (eigenvector)  matched  filter 

detection.  Conversely,  the  watermark  induced  image  distortion  V  is  minimized  for  any  given  target  bit 
error  rate. 

If  necessary,  further  bit  error  rate  improvements  below  Q  ^  yj \L+a~  )  ^or  any  fixed  distortion  T>  can 

be  attained  via  error  correcting  coding  of  the  information  bits  at  the  expense  of  reduced  information 

bit  payload.  The  maximum  possible  payload  in  bits  that  still  allows  -theoretically  for  asymptotically 

large  number  of  image  blocks  P-  message  recovery  with  arbitrarily  small  probability  of  error  is  CP 

where  C  =  max  7(6;  y)  is  the  Shannon  capacity  of  the  covert  link  in  bits  per  embedding  attempt.  We 
fb 

recall  that  7(6;  y)  identifies  the  information  conveyed  about  the  embedded  bit  6  by  the  received  vector 
y  and  fb  denotes  the  bit  probability  distribution  function.  For  Gaussian  host  data  x  ~  _A7(0,R;e)  and 
an  average  image  distortion  constraint  V,  we  can  calculate  [23] 

C  =  i  log  det  (lL  +  V  (Rx  +  a\ IL)  1  ssT )  (15) 

where  det(-)  is  the  determinant  operator.  We  can  show  that  the  signature  choice  s  =  q^  is  also  optimal 
in  terms  of  capacity,  i.e.  maximizes  the  capacity  C  of  the  covert  link  and  therefore  the  maximum 
allowable  payload  CP  for  the  host  vectors  xp  6  ML,  p  =  1, 2,  •  •  •  ,  P.  The  result  is  presented  in  the  form 
of  Proposition  3  below  whose  proof  is  given  in  the  Appendix. 


Proposition  3  Consider  additive  SS  embedding  according  to  (2)  with  x  ~  M  (0,  Rx).  Let  qi,  q2,  •  •  •  ,  qz, 
be  eigenvectors  of  Rx  with  corresponding  eigenvalues  X\>  \2>  ■  ■  ■  >  X  l-  For  any  watermark  induced 
distortion  level  V ,  a  signature  that  maximizes  the  covert  channel  capacity  is 


1 


sopt  =  arg  max  -  log  det  (lL  +  V  (R.x  +  0^1  l)  ssr  )  \  =  qL 


(16) 
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When  s  =  q/,,  the  covert  channel  capacity  is  maximized  to 

C(<\l)  =  _  log  (l  +  - - 77  )  bits  per  bit  embedding.  □  (17) 

2  V  AL+VnJ 

C.  Signature  Optimization  for  Linearly  Transformed  Host  Data 

In  this  section  we  generalize  the  previous  developments  and  assume  that  the  host  data  vector  x  can 
be  linearly  transformed  by  an  L  x  L  operator  of  the  form  I l~  csst  [18]  where  both  the  parameter  c  G  M 
and  the  signature  s  £  M.L,  ||s||  =  1,  are  to  be  designed2.  In  parallel  to  (2),  the  composite  signal  now 
becomes 

y  =  Abs  +  (I  l  —  cssT)x  +  n  (18) 

and  the  mean  squared  distortion  due  to  the  watermarking  operation  only  is 

V  =  E  |  ||A&s  +  (I l  —  cssT)  x  —  x||-|  =  E  j||  (Ab  —  cs1  x)  s||2|  =  A2  +  c2sTHxs.  (19) 

We  observe  that,  in  contrast  to  (3),  the  distortion  level  is  controlled  not  only  by  A  but  by  s  and  c  as 
well. 

With  signal  of  interest  Abs  and  total  disturbance  (1^,  —  cssT)x  +  n  in  (18),  the  linear  filter  that 
operates  on  y  and  offers  maximum  SINR  at  its  output  is 

E  1 1| wT (A6s)||2  j 

wmaxSiNR  =  arg  max — - - - - - -y  =  ((II  -  cssT)  Rx  (lL  -  cssT)  +  a2nlL)  s.  (20) 

w  E  j||wT((l£  —  cssT)x  +  n)||2  j 

The  exact  maximum  output  SINR  value  attained  is 

SINRmax  =  A2sT  ( (IL  -  cssT)  Rx  (IL  -  cssT)  +  a2nIL)  ^  s.  (21) 

In  the  following,  we  look  at  SINRmax  as  a  function  of  both  the  embedding  signature  s  and  the  parameter 
c,  SINRmax(s,  c),  and  identify  the  signature  and  parameter  values  that  jointly  maximize  the  SINR  at 
the  output  of  the  maximum  SINR  filter.  Our  findings  are  presented  in  the  form  of  a  proposition  below. 
The  proof  is  given  in  the  Appendix. 

Proposition  4  Consider  additive  SS  embedding  in  linearly  transformed  host  data  according  to  (18). 
Let  qi,q2,  -  -  ,  q l  be  eigenvectors  of  Rx  with  corresponding  eigenvalues  \\  >  A2  >  •  •  •  >  A l-  For  any 
watermark  induced  distortion  level  V,  a  (signature  s,  parameter  c)  pair  that  maximizes  the  output  SINR 
of  the  maximum  SINR  filter  is 

sopt  =  arg  max  |a2st  ((lL  -  cssT)  (lL  -  cssT)  +  cr2IL)  1  s|  =  qL  (22) 

2If  c  =  0,  we  revert  to  the  developments  of  Section  II. B.  If  c  =  1,  II  —  cssT  becomes  the  projector  orthogonal  to  s. 
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and 


copt  = 


Xl  +  +  F)  —  \J  (Al,  +  Cn  +  ^)2  —  4A iP 


2Al 


When  s  =  and  c  =  cOJ^,  the  output  SINR  is  maximized  to 

V  —  copt2\L 


SINRmax{(\rJ,copt)  = 


and  maximum  SINR  data  filtering  simplifies  to 


XL  (1  -  c°N)2  +  al 


,  -1 


The  target  distortion  V  is  achieved  when  the  bit  amplitude  is  set  at  A  =  yV  —  copt2\L 


(23) 

(24) 

>5 

cr 

III 

>5 

(25) 

c°Pt2XL. 

□ 

Proposition  4  shows  that  the  optimum  signature  assignment  is  still  the  “minimum”  eigenvector  of 
and  maximum  SINR  filtering  still  reduces  to  plain  signature  (eigenvector)  matched  filtering.  The 
optimum  selection  of  c  depends  on  the  minimum  eigenvalue  of  Rx,  A l,  the  noise  variance  a2,  and  the 
target  distortion  level  V.  The  optimum  pair  (sopt,copt)  allows  the  output  SINR  to  attain  its  maximum 
possible  value  • 

If  we  assume  that  x  is  Gaussian,  x  ~  A7(0,R.j;),  then  b  =  sign  (w^iaxSINRy)  is  the  optimum  bit 
detector  with  probability  of  error 

Pe  =  Q  (a^Jst  ((II  -  cssT)  Rx  (IL  -  cssT)  +  s)  =  Q  (v/simw^y)  .  (26) 

As  in  the  plain  additive  SS  embedding  scenario,  if  x  is  Gaussian  then  the  probability  of  error  is  a 
monotonically  decreasing  function  of  SINRmax.  The  pair  (sopt,copt)  which  maximizes  the  output  SINR 
of  the  maximum  SINR  filter  is  also  minimizing  the  probability  of  error  of  the  optimum  detector.  The 
details  are  given  in  the  following  proposition. 


Proposition  5  Consider  additive  SS  embedding  according  to  (18)  with  x  ~  M  (0,  Rx).  Let  qi,  q2,  •  •  •  ,  qz, 
be  eigenvectors  of  R^  with  corresponding  eigenvalues  X\  >  A2  >  •  •  •  >  A l-  For  any  watermark  induced 
distortion  level  T>,  a  (signature  s,  parameter  c)  pair  that  minimizes  the  probability  of  error  of  the  opti¬ 
mum  bit  detector  is 


and 


sopt  =  arg  min 

S 


cssT)  R^  (II  —  cssr)  +  a2 1] 


\  1 


q  l 


(27) 


Xl  A  cr2  T>  —  \J (A l  +  Cn  +  F)2  ~  4A lT 

2A ~L 
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(28) 


When  s  =  and  c  =  copt,  the  probability  of  error  of  the  optimum  detector  is  minimized  to 

-fe(qL)  C°pt)  =  Q 


V  -  copt2X, 


Al  (1  -  c°ptf  +  al 


(29) 


and  optimum  detection  reduces  to  b  =  sign(qj(y). 


□ 


Proposition  5  implies  that  the  “minimum”  eigenvector  of  the  host  data  autocorrelation  matrix  when 
used  as  the  embedding  signature  together  with  copt  allows  message  recovery  with  the  minimum  possible 
probability  of  error  Q  ,!pty^  ■>  1  (conversely,  the  induced  distortion  T>  is  minimized  for  a  given 

target  probability  of  error  Pe).  We  can  show  that  for  Gaussian  host  data  x,  the  covert  channel  capacity 
is  given  by 


C  =  i  logdet  (lL  +{V  -  c2sTB,xs)  ((II  -  cssT)  Rz  (IL  -  css)  +  IL)  1  ss2 


(30) 


Then,  we  can  prove  that  the  (sopt,  copt)  assignment  of  Proposition  5  is  also  optimal  in  terms  of  capacity. 
This  result  is  summarized  in  the  following  proposition  whose  proof  is  given  in  the  Appendix. 

Proposition  6  Consider  additive  SS  embedding  according  to  (18)  with*.  ~  AT  (0,TIX).  Let  qi,q2,--' 
be  eigenvectors  of  Rz  with  corresponding  eigenvalues  X\  >  X2  >  •••  >  A  l-  For  any  watermark  induced 
distortion  level  V,  a  (signature  s,  parameter  c)  pair  that  maximizes  the  covert  channel  capacity  is 


sopt  =  arg  max 

S 


and 


|lL  +  (T>  -  c2stRxs)  ((lL  -  cssT)  Rx  (IL  -  css)  +  u^Il)  1ssrj  =  qL  (31) 
Al  +  u2  +  T>  —  \J  (Al  +  u2  +  P)2  —  4A  lT> 


copt  = 


2Al 


When  s  =  qL  and  c  =  copt,  the  covert,  channel  capacity  is  maximized  to 

bits  per  bit  embedding.  □ 


c-v  ovt,  1,  V-copt2  Al 

C(qL,cp)  =  -log  [l  +  A£(1_c<yt)2  +  g2 


(32) 


(33) 


3. 


Multi-Signature  Embedding 


We  may  generalize  the  signal  model  in  (2)  to  cover  multi-signature/multi-nressage  embedding  of  the 
form 

K 

y  =  AbiSi  +  x  +  n  (34) 


i=  1 
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where  bits  £>i,  £>2,  •  •  •  ? &  K",  coming  potentially  from  K  distinct  messages,  are  embedded  simultaneously  in 
x  with  corresponding  amplitudes  Ai  >  0  and  embedding  signatures  s,;  6  RL,  ||sj||  =  1,  i  =  1,2, ... ,  K. 
Thus,  the  contribution  of  each  individual  embedded  message  bit  bi  to  the  composite  watermarked  signal 
is  AibiSi  and  the  mean  squared  distortion  to  the  original  host  data  x  due  to  the  embedded  message  i 
alone  is 

Vi  =  E  {WAAsiW2}  =  Al  i  =  1,2, . . .  ,  K.  (35) 

Under  a  statistical  independence  assumption  across  message  bits,  the  mean  squared  distortion  of  the 
original  image  due  to  the  total  multi-message  watermark  is 

K  2  'j  K 

T:  AibiSi  \=STJAi2.  (36) 

i= 1  J  i= 1 

With  signal  of  interest  AjbjSj  and  total  disturbance  j)Li  Aj6jSj  +  x  +  n  in  (34),  the  linear  filter 

j 

that  operates  on  y  and  offers  maximum  SINR  at  its  output  is 


WmaxSINRJ  =  R/^S  j  (37) 

where  R /3  is  the  “exclude-/’  data  autocorrelation  matrix,  that  is  the  autocorrelation  matrix  of  the 
disturbance  to  message  j  defined  as 

(k  \T)  a- 

^A:feiSj  +  x+n  >  =  ^2  Ai2SiSiT  +  Rx  +  a2IL.  (38) 

i=  1  J  i=  1 

The  exact  maximum  output  SINR  value  attained  is 


SINRmaxj  =  AjsJU/jSj.  (39) 

As  in  Section  II  for  single-message  embedding,  we  propose  to  view  SINRmaxj  as  a  function  of  the 
embedding  signature  s j,  SINRmaxj(sj),  and  identify  the  signature  vector  that  maximizes  the  SINR 
value.  Our  findings  are  presented  in  the  form  of  Proposition  7  below  that  parallels  the  developments  of 
Proposition  1  for  the  single-message  case. 


Proposition  7  Consider  additive  SS  embedding  according  to  (34)-  Let  qi,  q2,  •  •  •  qz,  be  eigenvectors  of 
R/i  in  (38)  with  corresponding  eigenvalues  X\  >  \2  >  ■  ■  ■  >  X l-  For  any  message  induced  distortion 
level  Vj,  a  signature  that  maximizes  the  output  SINR  of  the  maximum  SINR  filter  w maxsiNR,j  is 

s°pt  =  arg max  |a^stR^1s|  =  qL.  (40) 
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When  Sj  =  q l,  the  output  SINR  is  maximized  to 


SINRr 


;( qx-)  = 


Xl  +  O'n  Xl  + 


and  maximum  SINR  data  filtering  simplifies  to  wlnaxsmR  jS  =  q^Ry^y  =  q|[y. 


(41) 

□ 


In  summary,  Proposition  7  says  that  the  “minimum”  eigenvector  of  the  disturbance  autocorrelation 
matrix  when  used  as  the  embedding  signature  allows  the  output  SINR  to  attain  its  maximum  possible 
value  •  At  the  same  time,  maximum  SINR  filtering  becomes  plain  signature  (eigenvector)  matched 

filtering. 

For  fixed  bit  amplitude  values  A{  =  \fiDl  and  arbitrary  signature  initialization  s*  €  M^,  ||s,;||  =  1,  i  = 
1,2,...,  K.  consider  repeated  applications  of  Proposition  7  for  j  =  1,2,...,  K.  In  such  an  eigen-update 
signature  cycle  each  signature  is  replaced  by  the  minimum-eigenvalue  eigenvector  of  the  disturbance 
autocorrelation  matrix  as  seen  by  the  message  corresponding  to  that  signature.  Once  all  signatures  are 
updated,  a  second  update  cycle  may  begin.  The  whole  procedure  may  continue  for  a  predetermined 
number  of  cycles  m  or  until  convergence: 


s  j(m)  <—  min  eigenvector  (R^-(m))  ,  j  =  1,2,...,  K,  m  =  1,2, ...  . 


(42) 


It  can  be  proven  that  convergence  of  (42)  is  guaranteed  and,  as  shown  in  a  code-division-multiple- 
access  (CDMA)  literature  context  [24],  at  each  cycle  the  generalized  total  squared  correlation  TSCg 
of  the  signature  set  S  =  [si,  S2,  . . . ,  s#],  TSCg( S)  =  trace  (SA2Sr  +  +  ct2Il)2  where  A  = 

diag  (A\,  A2, . . .  Ax),  is  non-increasing  3. 

If  we  consider  channel  coding  the  message  bits  before  embedding  and  assume  that  the  host  x  in 
(34)  is  Gaussian,  the  steganographic  system  determined  by  the  signature  matrix  S  is  a  special  case  of 
the  17-user  Gaussian  multiple  access  channel  with  average  input  distortion  constraints  [23],  [27].  For 
such  a  channel,  the  sum  capacity  Csum  (defined  as  the  maximum  sum  of  message  coding  rates  at  which 
messages  can  be  recovered  reliably  [27],  [28])  is  a  reasonable  criterion  of  quality  for  the  signature  set  S 
and  equals 

CmJS,  A2,  R*,  u2)  =  ^logdet  (IL+  {Rx  +  a2nIL)  ~1SA2S7)  .  (43) 

Minimization  of  the  TSCg( S)  metric  translates  to  maximization  of  Csum  [28].  However,  decrease  in 
TSCg  at  each  cycle  of  the  algorithm  in  (42)  does  not  necessarily  imply  increase  of  Csum  as  seen  for 
instance  in  [29], [30]  via  binary  signature  examples. 

3Yet,  there  is  no  guarantee  that  TSCg( S)  will  converge  to  its  minimum  possible  value  (global  minimum)  [25],  [26]. 
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Apart  from  global  optimality  limitations,  the  multi-cycle  multi-signature  optimization  procedure  in 
(42)  requires  re-calculation  of  the  disturbance  autocorrelation  matrix  and  eigen  decomposition  at  each 
step  of  each  cycle.  A  simple  low-cost  alternative  to  (42)  could  be  a  conditionally  optimal  single-cycle 
design  method  based  on  the  following  proposition. 


Proposition  8  Consider  additive  SS  embedding  according  to 

i 

y  =  Ai+1bi+1si+1  +  ^2  AibiSi  +  x  +  n.  (44) 

i=  1 

Let.  qi,q2,-''cfL  be  eigenvectors  of  R^  in  (1)  with  corresponding  eigenvalues  \\  >  A2  >  •  •  •  >  Xl-  If 
the  signatures  s i  =  1, 2, . . . ,  l  <  L,  are  eigenvectors  of  Rx,  then  for  any  watermark  induced  distortion 
level  T>i+i  =  A2+l  a  signature  si+i  that  maximizes  the  output  SINR  of  the  maximum  SINR  filter  for  the 
bit  of  interest  6;+ 1  subject  to  the  constraint  sLs *  =  0,  i  =  1,2, ...  I,  is 


opt 

sz+i =  arg 


-1 


max 

sz+i 

s  T  s—0 
1+  Is* 


1  4  2 

A+i^i+i 


R 


+  'y^JAjbj: 


i=  1 


sl+ 1 


where  q7  is  the  minimum-eigenvalue  eigenvector  of  Rx  available. 
When  s/_|_i  =  q^,  the  output  SINR  is  (conditionally)  maximized  to 

SlNR^ai)  = 

Aj  <Tn  A  j  +  <Jn 

and  maximum  SINR  data  filtering  simplifies  to 


=  W 


(45) 


(46) 


wmaxSINRy  =  Aj  (R*  +  V  =  <ljy. 


(47) 


If,  in  addition,  we  are  allowed  to  assume  that  x  Af(0,Kx),  then  the  optimum  detector  for  the  bit  of 
interest  bi+i  is  bi+ 1  =  sign( qjy)  with  probability  of  error  Pe(<lj)  =  Q  ^ y f 2+a’^)  ■  n 

As  a  simple  illustration  of  the  use  of  Proposition  8  for  conditionally  optimal  multi-signature  design, 
suppose  that  we  want  to  embed  in  the  host  data  vector  x  €  ML,  K  <  L  message  bits  b\,  62,  •  •  • ,  bj<  with 
fixed  corresponding  amplitudes  Ai,  A2, . . .  ,Ak  (mean  squared  distortions  V\  =  A\,  V-2  =  A|, . . . ,  Vk  = 
A2k)  and  signatures  si,S2, . . . ,  s k  to  be  chosen.  By  Proposition  1  of  Section  II,  we  set  si  equal  to  the 
bottom  eigenvector  of  R^,  Si  =  q^.  Given  Si  =  q^  and  under  the  constraint  that  we  search  for  an 
S2  orthogonal  to  si,  by  Proposition  8  we  design  S2  =  qL-i  which  is  the  next  available  eigenvector  of 
Rr  from  the  bottom.  Given  si  =  q^,  S2  =  qL-i  and  under  the  constraint  that  we  search  for  an  S3 
orthogonal  to  both  S2  and  si,  by  Proposition  8  we  assign  S3  =  q^_2.  We  continue  this  way  until  the 
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final  assignment  sk  =  <\l-k+\-  Once  again,  a  welcome  side  effect  of  this  conditionally  SINR  optimal 
signature  design  procedure  is  that  the  maximum  SINR  receiver  for  each  message  bit  bi,  i  =  1,2, ...  ,K, 
simplifies  to  a  matched  filter  and  requires  no  knowledge  of  other  system  parameters. 

For  (fixed)  unequal  embedding  amplitude  values  A\,A-2,  ■  ■  ■ ,  Ak,  the  exact  order  by  which  the  eigen¬ 
vectors  of  are  drawn  to  become  signatures  is  important  if  we  consider  the  sum  capacity  of  the 
steganographic  system.  We  can  show  that  a  necessary  condition  for  a  maximum  sum  capacity  solution 
under  the  constraint  of  eigenvector  assignment  is  that  the  ordering  of  the  bit  amplitudes  be  inversely  pro¬ 
portional  to  the  ordering  of  the  eigenvalues  of  the  corresponding  signature  eigenvectors.  This  statement 
is  given  below  in  the  form  of  a  lemma.  The  proof  can  be  found  in  the  Appendix. 

Lemma  1  Consider  additive  SS  embedding  according  to  (34)  with  K  <  L  and  let  qi,q2,-"  jRl  be 
eigenvectors  of  Hx  with  corresponding  eigenvalues  X\  >  A2  >  •  •  •  >  A l-  Without  loss  of  generality 
assume  that  0  <  A\  <  A2  <  ■  ■  ■  <  Ak-  Then, 

C sum  (s/C  —  qL  1  •  •  •  ,  &K—i  q L—i,  *  •  •  ;  qL— K+l)  L 

C sum  (s/v  —  qL  1  •  •  *  i  &K—i  —  q  L—j  ,  •  •  ■  ,  ® K—j  qL— i  i  ^1  —  qL  —  iT+1 )  ,  ^ 


If  we  generalize  our  approach  and  view  the  individual  amplitudes/distortions  as  design  parameters 
themselves,  then  we  can  search  for  the  optimal  amplitude  assignment  that  maximizes  sum  capacity 
subject  to  a  total  allowable  distortion  constraint  T>t  =  A'f  -  We  derive  the  optimal  amplitude 

values  in  the  lemma  below.  The  proof  is  given  in  the  Appendix. 


Lemma  2  Consider  additive  SS  embedding  according  to  (34)-  Let  qi,q2,-"  >  Ql  be  eigenvectors  ofHx 
with  corresponding  eigenvalues  X\  >  X2  >  ■  ■  ■  >  X l-  If  the  signatures  s ,  associated  with  the  message  bits 
bi  are  distinct  eigenvectors  ofHx,  s =  q„t ,  £  {1,  •  •  •  ,L},  i  =  1,2,- ••  ,K,  then  the  sum  capacity  is 

maximized  subject  to  an  expected  total  distortion  constraint  Vt  if 


A 2  —  (—  (XVi  +  +  p)  (48) 

where  (x)+  =  max(x,  0)  and  p  is  the  Kuhn-Tucker  coefficient  [23]  chosen  such  that  the  distortion 
constraint  T>t  =  Y)i=i  A(  is  met.  □ 


To  find  the  necessary  parameter  value  p  in  (48)  we  suggest  to  arrange  the  participating  eigenvalues  XVi, 
i  =  1, 2,  •  •  •  ,  K,  in  ascending  order:  XZ1  <  XZ2  <  •  •  •  <  XZK .  Then, 


I-1  = 


J2i=l(^Zj  +  Q~n)  +  Tt 

J* 


(49) 
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where  the  cutoff  index  J*  is  the  greatest  integer  in  {J  :  J  £  {1,2,  •••  ,K }  and  — — — Zij  n - —  > 

\Zj  +  <r^}.  The  optimal  message  amplitude/distortion  allocation  solution  of  Lemma  2  can  be  viewed  as 
a  power  waterfilling  procedure  [23]  in  the  eigen  domain  of  the  host. 

Finally,  as  the  last  technical  development  in  this  paper  we  examine  the  possibility  of  carrying  out 
multi-signature  embedding  in  linearly  transformed  host  data.  We  assume  that  the  host  data  vector  x 
is  linearly  transformed  by  an  L  X  L  operator  of  the  form  1 i  —  c^sf  where  Cj  £  M  and  Sj  £  ML, 
||sj||  =  1,  i  =  1, 2,  •  •  •  ,  K,  are  the  parameters  and  signatures  to  be  designed.  The  final  composite  signal 
is 

K  /  K 

y  =  J2AibiSi+{lL-^cisiS 
i= l  \  i=i 

and  the  mean  squared  distortion  induced  by  each  individual  message  i,  i  =  1,2, ... ,  I\,  is 


x  +  n 


(50) 


Vi  =  E  j||(AA  -  Cjsfx)  Sj||2|  =  Af  +  cfsf  Rzs.j.  (51) 

With  signal  of  interest  AjbjSj,  the  autocorrelation  matrix  of  the  disturbance  is  R/j  =  A;2sis?:T  + 

cisisJ^j  Rr  (^-L  ~  cisisf^j  Acr^J-L-  Unfortunately,  in  contrast  to  (38)  for  multi-signature 
embedding  in  non-transformed  data,  R/j  remains  a  function  of  s j  (as  well  as  Cj ) .  In  this  context,  un¬ 
conditionally  optimal  multi-signature  multi-cycle  optimization  along  the  lines  of  (42)  is  practically  an 
unrealistic  objective.  Instead,  we  suggest  to  design  sequentially  the  amplitudes  Ai ,  parameters  Cj,  and 
signatures  s*  of  the  embedded  messages  i  =  1,2, ...  ,K  <  L,  such  that  the  output  SINR  is  conditionally 
maximized  given  all  past  fixed  embeddings  (single-cycle  optimization) .  Our  developments  are  presented 
in  the  form  of  Proposition  9  below  whose  proof  is  given  in  the  Appendix. 


Proposition  9  Consider  additive  SS  embedding  according  to 

AibtSi  +  I  1L  - 
i=i  \ 

Let  qi,q2,  ■  ■  ■  ,  q l  be  eigenvectors  o/Rz  in  (1)  with  corresponding  eigenvalues  X\  >  A2  >  •  •  •  >  A l-  As¬ 
sume  that  Si,  i  =  1,2, ...  ,1  <  L,  are  all  eigenvectors  of  Rx  and  j  is  the  index  of  the  minimum- eigenvalue 
eigenvector  of  Rx  available.  For  any  given  message  induced  distortion  level  Th+i,  an  (s^+1,q+i)  pair 
that  maximizes  the  output  SINR  of  the  maximum  SINR  filter  subject  to  the  constraint  s=_1s j  =  0, 


1 

y  =  Ai+ibi+isi+i  + 


E 

i=  1 


T 


q+is/+is^+1  x  +  n. 


(52) 


i  =  1,2, ...  ,1,  is 

s(+i  =  argnrax  Ai2sisiT+ 


1  + 1 


\i= 1 


iL-^CiSisf -Ci+1sws[+1  Kx\IL-^2ciSisf -Ci+1sws[+1  +a2nl L]  >  =  <ij  (53) 


i= 1 


i= 1 
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and  c°l\  =  arg  max  j  SINRmax( ,  q+i)| 


Aj  +  <+  +  Pj+i  —  (Aj  +  <+  +  T>1_ |_i)2  —  4AiP;+i 


2Ai 


(54) 


When  si+i  =  q j  and  q+i  =  f/ie  output  SINR  is  (conditionally)  maximized  to 


SINRmax(qj ,  0°^ )  = 


2^+1  -  c 


opt  2  X 

7+1  A? 


Ao  ( 1  —  cj 


opt 

l-\- 1 


(55) 


+  0+ 


and  maximum  SINR  data  filtering  simplifies  to  qjy.  If,  in  addition,  we  are  allowed  to  assume  that 
x  Af  (0,  Rx),  then  the  optimum  detector  for  the  bit  of  interest  bi+\  is  bi+\  =  sgn(qjy)  with  probability 
of  error 


Pe(WWil\)  =  Q 


t 


®<+i  -  c”S2A 3 


□ 


Ao(  1  —  c 


opt 

/+1 


+  CT- 


(56) 


As  an  illustrative  example  of  the  use  of  Proposition  9,  suppose  that  we  would  like  to  embed  in 
the  host  data  vector  x,  K  <  L  message  bits  b\,  62,  •  •  • ,  bx  with  individual  corresponding  mean  squared 
host  distortion  Vi,X>2 We  first  design  the  system  parameters  Si,  ci,  and  A\  for  message 
bit  b\  in  the  absence  of  any  other  message  in  the  host  image.  According  to  Proposition  9  ( l  =  0), 
the  optimal  parameter  selection  is  si  =  q^,  c\  =  xr+(rn+'Di~\/{xL+^n+'Di)  -4Al£>i^  ancj  by  (51)  A\  = 
\JT>\  —  c\\l.  Next,  we  proceed  with  the  second  message  bit  62  and  optimize  S2  and  C2  subject  to 
the  desired  distortion  level  V2  and  the  constraint  s^si  =  0.  Since  si  is  already  an  eigenvector  of 
Rx,  Proposition  9  (l  =  1)  offers  S2  =  qL-i,  C2  =  Xl^1+C7^+t>2~^  P^-^+^+T>2'>2  ~4:Xl^1'Ri  ^  anc[  by  (51) 
A2  =  \JV-2  —  c^Xl-i-  We  continue  calculating  signatures  s j,  parameters  c,,  and  amplitudes  +  as  above, 
always  subject  to  the  desired  distortion  T>i  and  orthogonality  between  the  signature  to  be  designed  and 
all  other  prior  signatures.  Provided  that  K  <  L,  the  final  set  of  designed  parameters  is  s^-  =  q.L-(K-i)> 

Ai_(K_1)+o-2+DK-v/(Ai_(x_1)+(T2+X)if)  -i\L_(K-i)VK  a  a  — ; 

CK  =  - 2A l-(k-i) - ’  alld  Ak  =  \JVk  ~  cKXL-(I<-  1)- 

Optimal  distortion  allocation  for  sum  capacity  maximization  in  multi-message  embedding  in  linearly 
transformed  host  data  subject  to  the  constraint  that  all  messages  are  assigned  distinct  eigenvectors 
of  Rx,  Si  =  q?)i ,  and  subject  to  a  total  distortion  constraint  T>r  =  X)*=i  =  X)?>=i  (+  +  Ci\Vi) , 

Vi  G  {1,  •  •  •  ,  L},  i  =  1, 2, . . .  K,  is  a  joint  optimization  problem  with  respect  to  c*  and  At.  We  suggest  an 
iterative  solution  approach  based  on  Proposition  9  above  and  Lemma  2  presented  earlier  in  this  section. 
We  initially  fix  the  distortions  induced  by  each  message  and  find  the  optimum  c*  parameters  according 
to  Proposition  9.  Then,  we  perform  optimum  amplitude  allocation  according  to  Lemma  2.  Based  on 
this  allocation,  we  re-evaluate  all  Ci  by  Proposition  9.  We  continue  until  convergence  is  observed. 
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(a)  (b) 

Fig.  2.  (a)  Aircraft  image  example  (512  x  512  grey-scale),  (b)  Aircraft  image  after  20 dB  (sopt,  copt) 
embedding  of  4 Kbits  and  additive  white  Gaussian  noise  of  variance  3 dB. 

4.  Experimental  Studies 

To  carry  out  an  experimental  study  of  the  technical  developments  presented  in  the  previous  sections,  we 
consider  as  a  host  example  the  familiar  grey  scale  512  x  512  “Aircraft”  image  in  Fig.  2(a)  that  has  been 
used  widely  in  the  pertinent  literature.  We  perform  8x8  block  DCT  single-signature  embedding  over 
all  bins  except  the  dc  coefficient.  Hence,  our  signature  length  is  L  =  63  and  we  embed  =  4,096 
bits.  For  the  sake  of  generality,  we  also  incorporate  white  Gaussian  noise  of  variance  =  3dB.  Fig.  3 
shows  the  recovery  bit-error-rate  (BER)  under  signature  matched  filter  detection  as  a  function  of  the 
distortion  created  by  the  embedded  message  over  the  0  to  20 dB  range  for  four  different  embedding 
schemes:  (a)  SS  embedding  with  an  arbitrary  signature,  (b)  SS  embedding  with  an  arbitrary  signature 
and  optimized  selection  of  the  host  data  transformation  parameter  c  as  in  [18]  (known  as  “improved 
spread-spectrum”  or  ISS),  (c)  SS  embedding  with  an  optimal  signature  according  to  Proposition  1,  and 
(d)  SS  embedding  with  a  jointly  optimal  signature  and  host  data  transformation  parameter  (sopt,  copt) 
according  to  Proposition  4.  The  demonstrated  BER  improvement  of  our  joint  signature  and  parameter 
optimization  procedure  in  particular,  measures  in  orders  of  magnitude.  Fig.  2(b)  shows  the  Aircraft 
image  after  20 dB  (sopt,  copt)-embedding  of  the  4 Kbit  message  and  3 dB  additive  white  Gaussian  noise 
disturbance  in  the  block  DCT  domain. 
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In  Fig.  4,  we  repeat  the  same  experiment  of  Fig.  3  on  the  256  x  256  grey  scale  Baboon  image  in  Fig. 
1(a)  (signature  length  L  =  63,  hidden  message  of  =  1,024  bits,  and  additive  white  Gaussian  noise 
disturbance  of  variance  3 dB).  Comparatively  speaking  (Aircraft  versus  Baboon  host  or  Fig.  3  versus 
Fig.  4  results),  message  recovery  for  the  Baboon  host  appears  to  be  a  somewhat  more  difficult  problem. 
Yet,  the  proposed  joint  signature  and  host  transformation  parameter  optimization  scheme  maintains 
a  near  10~10  BER  at  20 dB  host  distortion  and  outperforms  the  proposed  signature-only  optimization 
scheme  by  about  eight  orders  of  magnitude. 

In  Fig.  5,  we  continue  our  experimental  work  with  the  Baboon  host  and  plot  the  capacity  versus 
distortion  performance  curves  for  the  four  embedders  under  consideration.  We  see,  for  example,  that  at 
20 dB  host  distortion  the  jointly  optimized  embedder  offers  0.8  information  bit  payload  per  embedded 
bit.  This  number  goes  down  to  0.35  for  signature  only  optimization,  0.18  for  ISS  embedding  [18],  and 
0.15  for  arbitrary  signature  embedding. 

Next,  we  consider  the  problem  of  multi-signature  embedding.  We  keep  the  Baboon  image  as  the  host 
and  wish  to  hide  K  =  15  data  blocks/nressages  of  length  1, 024  bits  each  with  each  block/nressage  having 
its  own  individual  embedding  signature.  Each  message  is  allowed  to  cause  the  same  expected  distortion 
to  the  host  Vi  =  D,  i  =  1, _ ,  K.  Therefore,  for  statistically  independent  messages,  the  total  distortion 
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Fig.  6.  BER  as  a  function  of  the  per-message  distortion  D  (baboon  image,  K  =  15  messages  of  size 
1,  024  bits  each,  =  3 dB). 

to  the  host  is  =  KD.  As  before,  for  the  sake  of  generality,  we  add  to  the  host  white  Gaussian 

noise  of  variance  3 dB.  We  study  five  different  multi-signature  embedding  schemes:  (a)  Embedding 
with  arbitrary  signatures,  (b)  ISS  embedding  [18],  (c)  multi-cycle  eigen-signature  design  by  (42),  (d) 
conditional  optimization  by  Proposition  8  (sequential  s°pt,  q  =  0  assignment,  i  =  1,...,15),  and  (e) 
conditional  optimization  by  Proposition  9  (sequential  s°pt,  c°pt  assignment,  i  =  1,...  ,15).  As  seen  in 
Fig.  6,  the  superiority  of  the  latter  approach  (design  by  Proposition  9)  is  evident.  In  fact,  under  joint 
sequential  s°pt,  c°pt  optimization  even  the  least  favored  message  ( i  =  15)  outperforms  in  recovery  BER 
the  most  favored  (i  =  1)  message  under  sequential  s°pt,  q  =  0  signature-only  optimization  or  multi¬ 
cycle  eigen-signature  design  for  per-message  distortion  values  above  18 dB.  Fig.  7  shows  the  Baboon 
image  after  embedding  the  fifteen  messages  (15  •  1,  024  bits)  via  joint  sequential  s°pt,  c°pt,  i  =  1, . ... . ,  15, 
optimization  with  20 dB  per-message  distortion  (31.8 dB  total  distortion)  and  3 dB  variance  additive 
white  Gaussian  noise. 

Finally,  in  Fig.  8  we  present  sum  capacity  results  when  the  two  proposed  schemes,  sequential  s°pt 
design  and  sequential  joint  s°pt,  c°pt  design,  i  =  1, . . . ,  15,  employ  waterfilling  power  allocation  (use  of 
Lemma  2  alone  or  coupled  use  of  Proposition  9  and  Lemma  2,  correspondingly).  We  see,  for  example, 
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Fig.  7.  Baboon  image  after  multi-signature  embedding  via  Proposition  9  ( K  =  15  messages  of  size 
1,024  bits  each,  per-message  distortion  20<iE>,  a\  =  3 dB). 


Total  distortion  (dB) 


Fig.  8.  Sum  capacity  versus  distortion  (baboon  image,  K  =  15,  a ^  =  3 dB). 
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that  at  32 dB  total  distortion  the  waterfilled  s°pt,  c°pt  design  offers  information  bit  payload  of  about  11 
bits  per  15  bits  embedded,  while  the  waterfilled  s°pt,  a  =  0  design  offers  only  about  4  bits  per  15  bits 
embedded. 

5.  Conclusion 

We  considered  the  problem  of  hiding  digital  data  in  a  digital  host  image  (or  audio)  via  spread-spectrum 
embedding  in  an  arbitrary  transform  domain.  We  showed  that  use  of  the  minimum-eigenvalue  eigen¬ 
vector  of  the  transform  domain  host  data  autocorrelation  matrix  as  the  embedding  signature  offers  the 
maximum  possible  SINR  under  linear  filter  message  recovery  and,  conveniently,  does  so  under  plain 
signature  correlation  (signature  matched  filtering).  If  we  allow  ourselves  the  added  assumption  of  (col¬ 
ored)  Gaussian  transform-domain  host  data,  then  we  see  that  the  above  described  system  as  a  whole 
becomes  minimum  probability  of  error  and  maximum  Shannon  capacity  optimal  as  well. 

To  take  these  findings  one  step  further,  we  examined  SS  embedding  in  transform-domain  host  data 
that  are  modified  by  a  parametrized  projection-like  linear  operator.  We  found  the  joint  signature  and 
parameter  values  under  the  optimality  scenaria  mentioned  above.  Conveniently,  the  jointly  optimal 
signature  is  still  the  minimum-eigenvalue  eigenvector  and  the  SINR  optimal  linear  filter  at  the  receiver 
side  is  still  the  signature  correlator.  Yet,  joint  signature  and  parameter  optimization  was  seen  to  offer 
dramatic  improvements  in  SINR,  probabilty  of  error,  and  capacity  (Figs.  3,  4,  and  5,  for  example). 

Finally,  we  extended  our  effort  to  cover  multi-signature/multi-message  embedding.  First,  under 
signature-only  optimization  we  developed  a  computationally  costly  multi-cycle  eigen-signature  design 
scheme  based  on  the  disturbance  autocorrelation  matrices.  The  alternative  suggestion  based  on  the  host 
data  autocorrelation  matrix  alone  and  sequential  (conditional)  eigen-signature  optimization  is  practi¬ 
cally  much  more  appealing.  A  waterfilling  amplitude  assignment  algorithm  was  developed  as  well  to 
maximize  sum  capacity  under  eigen-signature  designs.  All  multi-signature  findings  were  generalized  to 
cover  parametrized  projection-like  modification  of  the  host  data  with,  once  again,  dramatic  improve¬ 
ments  in  probabilty  of  error  or  sum  capacity  (as  seen  in  Figs.  6  and  8,  for  example). 

As  a  brief  concluding  remark,  image-adaptive  signature(s)  or  signature(s)/parameter(s)  optimiza¬ 
tion  as  described  in  this  work  can  be  carried  out  over  a  set  of  host  images  (frames)  if  desired.  The 
only  technical  difference  is  the  calculation  of  the  host  data  autocorrelation  matrix  which  now  has  to 
extend  over  the  whole  host  set.  As  long  as  the  cumulative  host  autocorrelation  matrix  is  not  constant- 
value  diagonal  (/  al),  significant  gains  are  to  be  collected  over  standard  non-adaptive  SS  embedding 
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techniques. 
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Appendix 

Proof  of  Proposition  3 

We  need  to  find  s  that  maximizes  C  in  (15)  subject  to  ||s||  =  1.  Since  log(-)  is  a  strictly  monotonic 
function, 


sopt  =  argmax  det  (I l  +  'D  (Rx  +  <t2Il)  ss7  j  =  argmax  det  (  —  (Rx  +  <7^1  £)  + 


|| s|| =1 


i 


^2  j  \  1  c-.c~,T 

act  \  ll  t  ls  T  c 

sGM1',  ||s||  =  1 

Using  the  rank-one  update  rule  [33] , 

det  (Rx  +  <t2Il)  +  ssT)  =  det  (Rx  +  <x2I l)  (l  +  PsT  (Rx  +  <t2Il)  1  sj .  Therefore. 

sopt  =  argmax  (l  +  VsT  (Rx  +  a2nlL)  1  s')  =  qL.  □ 
seRC  ||s||=i  v  ' 


ss 


(57) 


(58) 


Proof  of  Proposition  4 

For  a  target  distortion  value  T>,  the  term  A2  in  (21)  equals  T>  —  c2sTRxs  and  is  maximized  for  s  =  q^. 
We  will  show  that  the  second  term  in  (21),  sT  ((i^  —  cssr)  Rr  (I/,  —  cssT)  +  a2  1l)  1  s,  is  maximized 
by  s  =  q^  as  well.  By  the  matrix  inversion  lemma, 

,  -l 


((IL  -  cssT)  Rx  (1L  -  cssT)  +  a2IL) 


1 


1 


1 


=  — 2  II  -  -jl l  (I l  -  cssT )  l  Rx  1  +  -A  (II  -  cssT)2  )  (IL  -  cssT)  and 

(J  m  (Tr,  \  (Tm  ) 


-1 


(59) 


T\ 2 


Ra,  1  +  — ~  (II  -  cssT) 
<7, 


~  (  R-x  1  +  ~2  lL 

U  nr. 


-1 


-1 


(c2  —  2c)  ( Rx  1  +  ^-1 L  )  ssT  ( R.r 1  +  I 


-1 


-1 


cq 


+  (c2  —  2c)  sT  (Rx1  +  Xl 


-1 


(60) 


Combining  (59)  and  (60)  we  obtain 

1  1  <72  -  sT  (Rj;1  + -4-Il  ’ 

sT  ( (IL  -  cssT)  Rx  (IL  -  cssT)  +  O* lL) -1  s  =  4  - V  /  y  x  ! 

^  a2  +  (c2  -  2c)  sr  (R71  + 

The  derivative  of  the  righthandside  of  (61)  with  respect  to  sT  ^R”1  +  s  gives 

1  ~an  (!  +  c(c—  2)) 


(61) 


C7‘ 


+  c(c  -  2)  ST  (Ra,1  +  -4l 


-1 


2  — 


<0  VcG 


(62) 


Hence,  sT  ((1^  —  cssT)  Rx  (II  —  cssT)  +  <r2I l)  1  s  is  a  decreasing  function  of  sT  ^R^,1  +  ^-IlJ  s. 

-1 


Yet,  <  sT  ( Rz  1  + 


arg  max 

s 


S  < 


Aj_ 


1+- 


Therefore, 


|st((Il-csst)  Rx  (Il-csst)+ct2Ip)  =  qL. 


(63) 
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Since  sopt  =  for  any  c, 

max  {SINRmax}  =  max  \  max{SINRmax}  >  =  max 

c,s  els  )  c 


v  -  c2\l 


(64) 


On  +  Al  +  c(c-  2)\L 

By  direct  differentiation  of  the  final  expression  in  (64)  and  root  selection,  we  obtain  copt  in  (23).  □ 

Proof  of  Proposition  6 

For  the  signal  model  in  (18),  the  channel  capacity  is 


C  = 


1  det  (to  -  c2 sTRxs)  sst  +  (lL  -  css1)  Rx  (lL  -  css1)  +  u2 II) 


(65) 


Applying  a  rank-one  update  [33]  to  the  determinant  in  the  numerator,  we  obtain 

<7  =  ±  log  (l  +  (to  -  c2srRxs)  sT  ((lL  -  cssT)  Kx  (IL  -  css)  +  a2nlL)  1  s)  .  (66) 

The  result  follows  from  the  proof  of  Proposition  4.  □ 

Proof  of  Lemma  1 

The  sum  capacity  of  the  channel  in  (34)  provided  that  all  signatures  s*  associated  with  the  message  bits 
bi  are  distinct  eigenvectors  of  R,  (s*  =  qVi,  u,  G  {1, 2,  •  •  •  ,  L},  i  =  1, 2,  •  ■  ■  ,  K)  is  given  by 
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K 


Csum  —  cy  |  (  1 


A2 


3= 1 


•  +  (j\ 


(67) 


Then,  Csum  (&K  —  qL;  ■  ■  ■  ■  SK—i  —  QL— ii  ■  ■  ■  >  Si  —  qZy — /t  -I- 1 )  Csum  (sk  —  QLi  ■  ■  •  j  &K—i  —  (JL— j  j  •  •  •  j 


Sftr-j  =  q  L-i 
Al  , 


,  Si  =  qL_K+i)  =  i  log 


A2 

1+  K-‘, 
^L—i+an 
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jK-i 


^L-j+Vr, 
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A2  . 

K—i 

^  L-  i  +A 


1+- 


K-j 


^L-i+ar, 


But’  < 1  +  xZ^k 


1  +  T  Akli: 


1  + 


□ 


■  <  ak. 

Proof  of  Lemma  2 

To  identify  the  set  of  amplitudes  (or  equivalently  distortions)  that  maximizes  the  concave  sum  capacity 
function  in  (67)  subject  to  the  distortion  constraint  T>t,  we  differentiate  the  Lagrange  functional 

1  K  (  A2  \  K 

£{Ai,  A2,  ■■■  ,  Ak)  =  -  log  1  + 


3= 1 


AVj  +  a2 


(68) 


3= 1 


with  respect  to  Aj  and  obtain 

A2  =  (—  (A^.  +  (T2)  +  fj,)  (69) 

where  //  =  is  the  Kuhn- Tucker  coefficient  [23]  chosen  such  that  T>t  =  Substitution  of 

the  optimal  amplitude  allocation  of  (69)  in  (67)  gives  the  maximum  attainable  sum  capacity  value 
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Csum  =  \  log  l-iJt/UU(^  +  an)  where  //,  J*,  and  \Zj  are  as  in  (49). 

Proof  of  Proposition  9 

The  output  SINR  of  the  maximum  SINR  filter  for  the  signal  model  in  (52)  is 
SINRmax  =  (P;+i-cf+1s^_1Rxs;+i)  • 


□ 


5*+ 1 


T/.  -  y^CjSjsf  -  Q+jSj-i-iS^!  j  Rx  ( Ij,  -  y^CjSjsf  -  Cj+iSj+iS^.!  j  +  CT^Il  +  TjSjsf  j  st+1. 


(70) 


2=1 


2=1 


Let  Q  =  [qi,  q2, . . . ,  qz,]  be  the  matrix  with  columns  the  eigenvectors  of  Rx,  A  =  diag  (Ai,  A2,  •  •  • ,  A £) 
the  diagonal  matrix  with  the  eigenvalues  of  Rx,  and  p  the  number  of  available  eigenvectors  of  Rx  that 
do  not  correspond  to  any  Sj,  i  =  1, . . . ,  l.  Since  Sj,  i  =  1, ...  ,1,  are  eigenvectors  of  Rr  we  can  write 

ct^Il  +  ^2  AiSisf  =  QAiQr  and  (lL  ~  ^CiSisf]  R^  (lL  -  =  QA2Qr  (71) 

i= 1  \  i=l  /  \  1=1  J 

where  Ai  and  A2  are  diagonal  matrices  of  dimension  L  x  L.  Consider  the  permutation  matrix  P  which 
partitions  Q  into  Q  £  Mixp  that  contains  the  available  eigenvectors  of  Rx  with  the  corresponding  eigen¬ 
values  in  descending  order  and  Q*  £  jj£x(L-p)  that  contains  all  used  eigenvectors:  QP3  =  Q  Q*  . 
Let  A  £  Mpxp  and  A*  £  mR-p)xR-p)  be  the  diagonal  matrices  that  contain  the  eigenvalues  of  the 
eigenvectors  in  Q  and  Q*,  respectively.  Then, 


(72) 


where  A^  and  A2  are  diagonal  matrices  of  dimension  ( L—p )  x  ( L  —  p ).  Since  Q  is  an  orthonormal  basis 
of  we  can  write  s/_|_i  as 


A  0 

ai  I„  0 

A  0 

papt  = 

,  PAxPt  = 

n  y 

,  and  PA2P7  = 

1 

O 

> 

* 

1 _ 

0  A*1 . 

1 

O 

to  * 

1 _ 

sz+i  — 


Q  Q*  ] 

P 

=  QPr 

P 

(3* 

(3* 

(73) 


where  (3  £  Mp  and  (3*  £ 


i-p 


The  constraint  sf+lSi  =  0,  i  =  1, 2, is  equivalent  to  (3*  =  0.  Hence, 


s/+i  =  QPJ 


P 

0 


(74) 


/3T  0 1 


The  unit  norm  requirement,  ||sj+i||  =  1,  implies  that 

^opt 
3«+i  — 

Substituting  s/+1  from  (74)  to  the  first  term  in  (70)  we  obtain 


pqtqpj 


p 

0 


=  1  or 


P 


=  1. 


We  want  to  find  s?j\  =  arg max {SINRmax}  subject  to  ||s/_|_i||  =  1  and  sLs j 

^  Si+1  “r 


Vi+1-cf+1sf+1Rxsi+1  =  Vi+ 1  -  cf+J 31  A/3. 


=  0,  i  =  1,2 


(75) 
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Using  (71)  and  (74),  the  second  term  in  (70)  becomes 


(3T07 


1-L  —  Cl+i 


P 

0 


~T 


P  CP 


PAPT  I  L-Cl+1 


p 

0 


pTo 1 


-P(A-Ai-A2)P2 


p 

0 


(76) 


which  using  (72)  reduces  to 


p1  0 2 


Ip-q+i/3/31^)  A^Ip-Q+i^/37^ +<t^Ip  0 

0  A*  +  A* 


-1 

p 

0 

(77) 


We  know  that  if  a  matrix  A  = 


An  A12 

A21  A22 

—  1  A  ^  /A  A  A  "1  A  \  — 7 


All  —  Ai2A221A2l)  (All  —  Ai2A221A2l)  -A-12  A22 

A22  —  A2lA111Ai2)  A2lA111  (A22  —  A2lA111Ai2) 

ther  to 


is  invertible  and  At  j1 ,  A22x  exist,  then  A  1  = 

[34],  Hence,  (77)  reduces  fur- 

(78) 


as 


PT  ((lp-Q+i/3/3T)  A  ( lp-ci+iPpT )  +  a2n Ip)  p. 

Using  (75)  and  (78),  the  initial  optimization  problem  can  be  written  equivalently 

P  =  arg  max  j  (t>i+i  -  c2+1pT  A, Pj  (  (ip  -  ci+iPff^ 


subject  to 


P 


A  (j-p  —  ci+iPP  )+o%Lp)  P 

~jp  _  _  ~jp  /  /  _ (j 

1.  We  will  show  that  both  terms  in  (79),  U>/+i  —  cf+1P  A/3  and  0  ( (ip  — q+i/3/3 


(79) 


Ip  —  ci+iP(3  ^  +<r^Ip^  P,  are  maximized  by  the  same  p.  For  the  first  term  we  have: 

arg  max  {Vl+i  ~  c2l+lpTkp]  =  arg  min  {pTkp)  =  minimum  eigenvector  of  A. 
Consider  now  the  second  term.  By  the  matrix  inversion  lemma, 


(80) 


Ip  -  ci+iPp1)  A  (ip  -  q+i/3/3T)  +  cr^Ip 


-1 


—  2  Ip  _4  Ip  (ip  cl+lPP 


~  1  /  ~  ~~T\  ^ 

A^1  +  —  (ip  -  ci+iPP 


-1 


Ip  -  ci+iPpJ 


(81) 


Using  the  matrix  inversion  lemma  again, 


~_i  1  /  ~  ~T\2 

A  +  —j  (ip  ~  ci+iPP 


-1 


=  A’1  +  -4 


,  +  1  ^  1  (CZ+ 1  -  2cU l)  (A  1  +  yrlp)  PP  (A  1  +  yrF 

0-2  +  (<f+1  -  2q+i)  pT  (a-1  +  Aflp)  P 


-1 


(82) 


26 


Combining  (81)  and  (82)  we  have 


pT  ((lp  -  c;+i/3/3T)  A  (lp  -  cm/3/f)  +  < 


,  N-1  -  1 

av3p  )  P  —  ~o 


an  ~  P  1  +  ^Tp)  P 
an  +  (c?+l  —  2c;+i)  (3  ^A_1  +  4j-I p'j  (3 


Differentiation  of  the  righthandside  of  (83)  with  respect  to  (3  f  A  1  +  4rIp')  (3  gives 


-<t*(1+c1+1(C,+1-2)) - <  o  Vq+i  g  M_  Hence,  ^ 


an~0  A  +— 2"Ip  0 


an~^cl+l(cl+l~ 2)/3  (  A  1  +  — 2-1  p  )  /3 


(  A  -1  _L  1  T  ) 


a  decreasing  function  of  (3  (  A  1  +  -\lp  )  (3.  However, 


*min 


°n+cz+i(cz+i  — 2)/3  |^A  1  +  -^-IpJ  0 


where  Amjn  and  Amax  are  the  minimum  and  maximum  eigenvalues  in  A.  Therefore, 


argmax|/3  y[\p  — ci+\[3j3  j  A  ^Ip  — q+i/3/3  j+<7^Ipj  /3  j  =  minimum  eigenvector  of  A.  (84) 


We  conclude  (cf.  (79),  (80),  (84),  and  (74))  that  the  SINR  expression  in  (70)  is  maximized  when  s;+i 
is  the  minimum  available  eigenvector  of  Rx  for  any  c\+\.  Hence, 

^max+i  {SINRmax}  =  max  jmax  |  (vi+1  -  cf+1^T Af3j  ^  ((lp  -  q+i/3/3T^)  A  (lp  -  ci+i/3/3T)  +  o£l (3 

_ 'Di+i  ~  c?+iAj _ 

an  +  A j  +  ci+i(ci+i  -  2) A j 

where  A  j  is  the  minimum  available  eigenvalue  of  (equivalently  A  j  is  the  bottom  element  of  A,  A  j  = 
A  min)-  The  optimum  q+i  value  can  be  computed  by  setting  the  derivative  of  the  last  expression  in  (85) 
equal  to  zero.  The  latter  gives  two  candidate  values  for  q+i-  We  select  the  value  that  maximizes  (85), 
which  is  the  one  in  (54).  □ 
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